About the Information Technology Resources Board (ITRB)

The ITRB was established in July 1996 by Executive Order 13011 to provide a peer assessment of mission critical information system projects by identifying critical issues and findings; to frame these in management and technical perspectives; and to recommend actions to mitigate risks or resolve issues. ITRB members are executives, and experienced practitioners from across Federal agencies who bring different program, technical, and acquisition management expertise to managing and developing major information systems. The ITRB’s activities promote measurable improvements in mission performance and service delivery through the strategic application of information technology.

The ITRB mission goals are:

• Assist agency management to leverage information technology to improve mission performance and service delivery.

• Recommend to agency management opportunities for improving how the agency develops, acquires, and manages major systems.

• Share information across government on critical issues and experiences from information technology assessments.

• Help the Office of Management and Budget analyze, track, and evaluate risks and results of major information technology investments.

Current ITRB Members:

Arnold Bresnick, Chair	Department of Agriculture
Valerie Wallick, Vice Chair	Department of the Navy
Sandra Borden	United States Coast Guard
Kenneth Byram	Federal Aviation Administration
Kevin Carroll	Department of the Army
Kay Clarey	Department of the Treasury
Wayne Claybaugh	Social Security Administration
Mary Ellen Condon	Department of Justice
Mark Day	Environmental Protection Agency
Joanne Ellis	Department of Agriculture
George Hyder	Office of Personnel Management
Ken Heitkamp	Department of the Air Force
Myron Kemerer	National Aeronautics and Space Administration
Mike Laughon	Department of the Interior
Jean Lilly	Internal Revenue Service
Eric Mandel	Department of Commerce
Emory Miller	General Services Administration
Staff:
Sandra Hense	General Services Administration
Ginni Schaeffer	General Services Administration
Jake Asma	General Services Administration

---

Contents

Introduction

What can be learned from the ITRB's unique insights?

Strategic Definition

Where is the organization going and how can technology help get there?

Vision

Mission

Customers

Technology

Organizational Leadership

Who is steering and what is the definition of success?

Chief Information Officer

Performance and Results-based Management

Capital Planning and Investment Control

Program/Project Management Responsibilities

Technology Management

How can technology help fulfill the mission?

Architecture Infrastructure

Acquisition of Information Technology

The ITRB Perspective

What has been learned from the ITRB's experiences?

Introduction What can be learned from the ITRB's unique insights? This document offers the combined strength of the ITRB members' extensive individual experience in delivering mission critical systems, and the Board's unique collective perspective drawn from peer assessments. The ITRB's assessments examined mission critical initiatives at various stages in the project life cycle, from conceptualization to product or service delivery. The lessons the Board learned about mission critical systems can contribute to successful outcomes across Federal technology initiatives. The framework for this document emerged from the ITRB assessment experiences and echos the emphases of recent legislation that redefines government management. Strategic Definition Where is the organization going and how can technology help get there? Articulate the agency vision to focus delivery of the mission. Identify and serve customers and stakeholders. Align the role of technology to serve the business objectives of the organization. Organizational Leadership Who is steering and what is the definition of success? Leaders can promote strategic alignment by focusing on pivotal management areas: the Chief Information Officer (CIO) role; performance and results-based management; capital planning and investment control; and program management responsibilities. Technology Management How can technology help fulfill the mission? Strategic technological alignment relies on definition of the architecture infrastructure and appropriate acquisition of information technology (IT). The following chart depicts the relative frequency of specific concerns that the ITRB encountered in its first four years. The chart does not represent the entire universe of Federal systems; rather, it represents only those systems assessed by the ITRB.	"Establish goals for improving the efficiency and effectiveness of agency operations and, as appropriate, the delivery of services to the public through the effective use of information technology." The Clinger-Cohen Act
Strategic Definition Where is the organization going and how can technology help get there? Vision Formulate an agency vision Agency leadership face the challenge of creating a clear, articulate vision of how the agency will accomplish its overall mission. Validate the vision with customers and stakeholders. Define the business processes necessary to achieve the vision. Translate the vision into reality Make the vision more than a slogan. It is not enough for the agency's leadership to set down its vision in a strategic planning document. Leadership should actively translate the vision into reality. Because information technology is not well understood by many, the value of technology solutions may be viewed as a matter of faith. Make sure that the vision is elaborated in a series of increasingly detailed, useable documents. Mission Execute the business mission Articulate an agency Strategic Plan based upon the agency's vision. Focus on a high level description of desired outcomes and critical success factors to gauge success of implementation. Integrate the planning and execution framework to link plans and processes. Create accountability Compose a Business Plan to identify services and functions the agency must perform to achieve outcomes defined in the Strategic Plan. Identify a management oversight process that assigns accountability for its implementation. Monitor implementation and use the capital investment process to assist in evaluation. Customers Identify and serve customers and stakeholders The agency vision should be readily shared and validated with stakeholders and customers. Cast a wide net when identifying customers and stakeholders; include those inside the agency as well as those outside of it. Actively communicate with customers and stakeholders Maintain an environment in which stakeholders and customers are able to raise important questions about the agency's future direction. Provide the means to receive regular, structured input. Create a sustained focus on constructive innovators. Identify competing priorities among these stakeholders. Create mutual expectations. In this way, the agency ensures that its vision remains aligned with the expectations of its customers and stakeholders. Technology Align the role of technology with the business of the agency Technology enables the agency to accomplish its business. In order to harness the potential of technology, drive capital planning and the IT architecture with the Business Plan. Define the target IT architecture in the context of the Business Plan. Use capital planning to assure progress toward that business-driven architecture. Preserve sufficient flexibility in the architecture to accommodate changes in both business processes and technology. Avoid the "bleeding edge" Keep a focus on business value, rather than the dazzle of technology. Deliver the agency's business priorities, and emphasize business solutions over technical sophistication. While technology may drive change to a business process, make certain that the change efforts are prioritized for the benefit they bring, not simply because they are technically attractive.	In a recent assessment, the ITRB found a project's scope had grown dramatically out of control. This was symptomatic of the disjuncture between information technology programs and the agency's strategic business vision. The situation was exacerbated by the agency's eclectic mission requiring a proactive approach to continuous communication with stakeholders, often with competing priorities. The ITRB recommended seeking a common set of expectations among these diverse constituencies as an important step in realigning IT with the agency mission. The ITRB recommended an agency adopt a multi-track approach to manage core mission critical production work, conduct strategic planning for future service delivery, and initiate processes to implement these future systems. One track should focus on pressing operational needs, without which the agency faced a serious risk of failure to maintain current operations at acceptable levels. The second track should develop a strategic project plan to incrementally realize the future service delivery business vision and a disciplined, rigorous implementation process to carry out the plan.
Organizational Leadership Who is steering and what is the definition of success? Chief Information Officer (CIO) Align the IT strategic vision that supports the agency's overall business strategy The CIO uses technology to help agency leaders achieve desired business outcomes. The CIO's focus on the strategic management of information ensures that systems align with the agency's organization structures, and deliver enhanced performance across the business enterprise. Enforce the IT Strategic Plan The needs of the enterprise are reflected in a Strategic IT Plan and delivered through disciplined implementation of common initiatives. Define and implement a target architecture that reflects the strategic intent of the agency. Monitor and enforce the implementation of the target architecture throughout the life cycle of systems Institutionalize the oversight and review of IT initiatives. Implement a formal inspection process to determine architecture compliance. Link architecture compliance to strategic and business objectives, and to the capital investment process. Centrally fund and manage key components of the architecture. Gain support of stovepipe functions Mission critical systems are often expected to integrate across culturally different, and often fiercely independent, functions in a single agency. The functions have their own budgets, constituencies, and legacy systems. Address functional concerns to win their support. Prioritize information technology investments Use the architecture as a framework for decisions on IT investments. Employ strategic planning, policy making, performance management, evaluation, and technical expertise to ensure balanced programmatic and technical perspectives on IT investments. Make sure the architecture guides incremental delivery of new or continuing functionality and transition of legacy systems according to funded priorities. Act as central point of accountability for technical decisions across IT projects agencywide Ensure adequate infrastructure to provide reliable, accessible enterprise IT services. Create a CIO organization to bring focus and accountability to enterprisewide technical decisions. Provide IT decision making assistance to executive and senior management Leverage CIO knowledge of architecture and performance requirements to inform senior level managers of issues concerning cost, schedule, risk, and expected benefits. Make sure the investment process evaluates how well IT initiatives support the enterprisewide business objectives. Ensure needed skills and knowledge are available to support IT projects The need to attract and retain highly qualified personnel is more critical than ever. Ensure appropriate skills and experience are developed or acquired to meet organizational needs. Performance and Results-based Management Link IT projects to agency goals and objectives Formalize planning processes and define project success from the outset in terms of specific business objectives. Prioritize business objectives according to their impact on the customer. Assign roles and responsibilities specifically for planning-related activities. Have the CIO help anchor project plans in the organization's business and IT plans. Reexamine systems over time to make sure that they continue to support shifting goals and objectives. Develop measurements in order to manage performance Focus on outcomes rather than outputs. Define the metrics for which project participants will be held responsible. Gain agreement on critical metrics and use them to drive planning and delivery. As systems are developed, make progress on critical metrics visible to agency leadership. Link business and technical requirements to desired outcomes Define the requirements that are essential for ensuring that the architecture ultimately supports desired outcomes. Pay particularly close attention to business priorities and technical interfaces. Develop a plan to ensure technical compliance with architecture standards. System requirements must be documented to ensure their consistency with the overall agency-level design. Institute an implementation and control methodology Formally capture and document agreements among all customers and stakeholders. Secure an informed agreement up front, and maintain this agreement throughout the system life. This will ensure that the system delivers expected results. This will also help align the system with the organization's business plans and supporting IT plans. Use the architecture to guide and control implementation. Work toward a systems implementation that will deliver, in twelve months or less where appropriate, incremental, useable levels of functionality which support specific business objectives. Implement technical and performance review processes One recommended technique is to outsource for independent verification and validation and (IV&V) support. For executive leadership, IV&V can enhance the project's accountability and help raise its visibility. For project managers, IV&V can provide unbiased perspectives and expertise on specific project elements. Perform periodic cost-benefit analyses and life cycle cost estimates. Use this information for go/no-go decisions at major system milestones and for technology portfolio investment decisions. Capital Planning and Investment Control Plan IT investments in support of agency business objectives Well-managed IT investments can contribute greatly to the cost effective achievement of an agency's mission. Capital planning provides a systematic approach to managing the risks and rewards of those investments. A Capital Plan arrays IT investments in a portfolio of selected assets needed to accomplish the organization's priorities and ensures that those investments are linked to business objectives. Execute the plan to integrate planning, budgeting, procurement, and management of IT and others assets, as defined by the agency's business objectives. Institute an investment review process A formal capital asset management infrastructure establishes lines of authority, responsibility, and accountability for the management of IT assets. An executive investment review board evaluates the agency's entire portfolio and makes decisions on the mix of assets needed to achieve business objectives within budget parameters. The board consistently tracks the selected initiatives in the portfolio. Gain cross-functional focus for investment decision making by creating a board comprised of senior operations executives from across the enterprise, including the CIO and Chief Financial Officer. Build a balanced view of agency initiatives with a representative board. The CIO ensures that IT initiatives which support enterprisewide business activities receive due consideration in the investment process. Determine and evaluate net benefits and investment risks for IT A formal investment management process offers a framework for consistent, informed decision making throughout the organization. Standardize quantitative and qualitative criteria for ranking IT investments and seek rigorous comparison among all competing opportunities. Gauge the costs and benefits of all initiatives against established criteria. Optimize the portfolio mix in order to manage risk and maximize return throughout the system life cycle. Align resources with business priorities The investment board evaluates competing investments, including IT, and recommends annual funding to balance the portfolio. Analyze the extent to which opportunities contribute to business priorities. Then prioritize among competing opportunities to effectively allocate limited funds. Use contract management as a tool for asset management Structure IT contracts into manageable, relatively short-term deliverables that can be easily evaluated and modified as necessary over time. Define critical milestones or events that require management decision before proceeding. Link the results of this decision making into overall portfolio decision making. Consider use of commercial off-the shelf technology and alternative technical approaches. These strategies provide a basis of comparison useful in investment decision making. Consider independent risk analysis of contract related IT decisions. Build in flexible contract features that allow for responsiveness and improvement throughout the contract life. Program/Project Management Responsibilities Align project goals with the business goals of the agency What accomplishments will be critical for the project to be considered successful? A clear definition of business success will ensure that project efforts support the agency's strategic plan. Define project boundaries at the outset, and use this definition to manage requirements throughout the project. Adopt a formal system lifecycle process The system life cycle provides an organizing structure for system development, enabling alignment of project objectives with appropriate technologies and resources. Different kinds of projects require different degrees of rigidity in the sequencing of their phases. Long, complex projects intended to modify familiar systems typically yield to more rigid sequencing. On the other hand, less rigid sequencing may be required to achieve a series of innovations under conditions of high uncertainty. Implement program/project management and incremental control processes Define requirements so that they may be thoroughly tested and validated according to a pre-established test plan. Identify frequent milestones with a defined set of measurable pass/fail performance criteria, and then structure related contracts so that they reflect the same units, granularity, and milestones. This will enable measurement of earned value throughout the contract life. Use configuration management to ensure that the system is developing in the intended fashion. Configuration management practices will illuminate how incremental changes to the system will impact desired results. Focus on cost, schedule,and quality of program/project delivery Conduct life-cycle cost analyses to evaluate the impact of design implementation alternatives. Use agreed upon plans to control the resources applied to the system. Periodically review actual project expenditures and compare them to the projected budget. Structure routine progress reports so that they highlight exceptions from anticipated progress. Assess risks explicitly throughout the system life cycle Every system entails risks. Know a system's major risks and manage them actively. A system may encounter uncertainties associated with the system's requirements, the technology that the system will employ, or the way that the system will affect business processes. Compressed development times may heighten risks. Public or political attention may also heighten risks. Articulate potential risks to stakeholders. Ensure decision making accountability and visibility Identify the key leaders who will be responsible for results. Ensure that project managers, program managers and executive leadership understand their respective responsibilities. Build an experienced project team, led by a seasoned project manager. Stress the importance of accountability by systematically introducing constructive criticism into current practices. A highly useful technique is to outsource for IV&V support. It is critical for executive leadership to listen to IV&V advice. Project participants need a clear idea of how well the project plan is working. Establish a set of key progress indicators and make them visible to all project participants. Structure the project in such a way that key leaders can make go/no-go decisions at highly visible milestones. Actively encourage participants to make challenges known before they become major problems. Manage up as well as down New political appointees in an agency may have difficulty understanding all aspects of the agency's work from day one. Furthermore, they may have a relatively brief time frame in which to deliver results, and their agendas and time horizons may differ from those of career government workers. New agency leaders need to be educated and brought into the development process. Successful delivery of mission critical systems requires a collaborative process between career and appointed officials.	The CIO "ensure(s) that information technology is acquired and information resources are managed" according to the business priorities of the organization. The Clinger-Cohen Act One ITRB-assessed project suffered confusion among contractor and agency personnel as to who made key decisions. In the absence of cooperative leadership, critical analysis of functional requirements was seriously lacking. The ITRB recommended the project not only clarify the respective roles of project team members, but that it reorganize its executive steering committee to make it truly accountable for all final project decisions. One ITRB project had a large shortage of staff on the agency management team. The ITRB recommended the management team take all possible actions to expand its staff, concentrating on the addition of computer software and systems expertise. ITRB also recommended that contract personnel be more effectively used to provide project management support. "Ensure that performance measurements are prescribed for information technology used by or to be acquired for, the ... agency and that the performance measurements measure how well the information technology supports programs of the ... agency" The Clinger-Cohen Act In one instance the ITRB found an agency's and contractors' recent adoption of earned value and software engineering approaches had started to make significant improvements in project management and provided a solid basis for decision making about information technology projects. In the majority of assessments it has conducted, the ITRB has recommended that organizations immediately establish a process for independent validation and verification and that executives explicitly consider IV&V recommendations when making decisions. "(D)esign and implement a process for maximizing the value and assessing and managing the risks of the information technology acquisitions of ... the agency" The Clinger-Cohen Act See the ITRB's publication, "Project Management for Mission Critical Systems, A Handbook for Government Executives". The ITRB recommends that agency executive leadership set, clarify, and monitor success in achieving business expectations by continually reviewing whether projects are on time and within budget; focusing resources on highest priority projects; overseeing projects to assure delivery of expected outcomes; and communicating expectations to project stakeholders. The ITRB assessed a project whose software development process was in a perpetual state of change. The ITRB recommended the establishment of configuration management baselines as well as cost and schedule baselines. The ITRB encountered one project which had suffered from a severe lack of agency oversight. The agency had not effectively monitored project deliverables, managed project risks, or controlled project resources. The ITRB recommended that the agency identify specific termination criteria for the project, and that the agency commit to terminating the project if it should in fact become necessary.
Technology Management How can technology help fulfill the mission? Architecture Infrastructure Define an IT architecture that aligns the requirements for information systems with the processes that support the agency's mission and goals The target IT architecture brings to life the requirements defined in the Business Plan and describes the relationships among the work the agency does, the information the agency uses, and the technology the agency needs. The architecture should be flexible enough to accommodate changes in business processes and technology over time. Rely on the target architecture as a framework for making sound business decisions. Make tradeoff decisions for implementation at the agency level to ensure that future business processes can be properly supported. Require compliance with the target IT architecture for Capital Plan approval. For example, align Year 2000 work with planned business outcomes articulated in the architecture to minimize future changes to legacy systems. Create an enterprise architecture that includes business processes, data descriptions and relationships, and technology infrastructure Document the relationships among business and management processes, and IT. Plan for incremental transition to an enterprise IT architecture to ensure continuity of support for business processes. An open system architecture enables incremental migration to new platforms and easier insertion of technology. Define corporately shared data and determine how best to share with internal and external systems. An adaptable, open architecture eliminates costly redundancy in data maintenance and development. Use short duration prototypes to test various elements for inclusion in the architecture. Prototype business processes, then implement agencywide. Implement a flexible architecture through a collection of standards by which the agency evaluates and acquires new systems A flexible, standards-based architecture provides for the common needs of related organizations. Standards ultimately guide and constrain IT asset acquisition. Use the architecture to assess and prioritize IT requirements. Establish enterprise protocols and core standards to integrate new systems into the current technical environment. Align standards to ensure enterprisewide compatibility while flexibly supporting each component's delivery of services. Regularly identify and prioritize the resolution of technical standards. Use the information technology architecture to ensure interoperability and security of information systems Architecture core standards ensure enterprisewide compatibility. Emphasize the importance of security standards for data and infrastructure reliability. Security is essential for dependable service and critical to protecting information assets. Security requirements should be addressed in the development of all parts of the architecture. Monitor architecture compliance Ensure compliance with architecture standards at all levels of the agency, consistent with the mission and functions of the organization. Clearly assign organizational responsibility for the core computing infrastructure. A Concept of Operations outlines the functions, roles, and responsibilities for implementation and management of the architecture. Implement a formal inspection process for architecture compliance. Establish a configuration control process. Enforce enterprise protocols and standards consistently. Enforce waiver criteria to secure compliance. Test the integration of new or modified architectural components. Acquisition of Information Technology Ensure acquisitions support strategic business objectives Take a strategic view of IT to design and manage the effects of a mission critical system's acquisition, development and deployment on the agency overall. Make sure that the strategic objectives drive the acquisitions associated with mission critical systems. Translate the strategic vision into clearly defined deliverables with measurable expectations and acceptance criteria. Align contract vehicle type and incentives to the nature of tasks Select the type of contract that creates the most productive incentives for the contractor and for the government. Establish performance incentives and measures that focus on each contract module. Different contract types may be needed for various modules. A firm, fixed price approach ensures that requested changes to requirements are thoroughly evaluated before implementation changes are made. This disciplined approach may be inappropriately stringent for exploratory development work that requires cutting edge technology and experimental innovation. Implement contract monitoring and control processes Develop major milestones, baselines, and approval processes to serve as the basis for proactive contract management. Establish formal decision points to allow for close control of progress. Tie deliverables to decision points and key milestones. Establish technical requirements for deliverables to definitize a contract. Define modular delivery in a contract to achieve interim operational capabilities. Closely monitor product delivery and quickly resolve issues. Ensure that all deliverables undergo acceptance testing. Tie progress payments to accomplishment of key milestones. Strengthen the role of contract management in the decision making process for mission critical systems to ensure that contractual direction and expertise is part of the management process. Assure acquisition strategies and contractual direction are incorporated into executive decision making The management process for mission critical systems provides explicit roles for affected stakeholders to monitor systems development and provide appropriate input. The executive steering committee accountable for final decision making should include individuals with contract and acquisition expertise. The project team should include the contracting officer as a member.	"(A)gencies shall significantly improve the management of their information systems, including the acquisition of information technology". Executive Order 13011 "(T)he term `information technology architecture' ... means an integrated framework for evolving or maintaining existing information technology and acquiring new information technology to achieve the agency's strategic goals and information resources management goals." The Clinger-Cohen Act One Federal organization found that its IT staff were constantly being pulled into short-term, high-profile projects for different offices. This resulted in a series of redundant systems architectures, and left little time to deal with enterprisewide concerns such as routine service, support and maintenance. The ITRB recommended that the organization establish a distinct IT special projects staff to react to short-term requests. An agency requested the ITRB review its enterprisewide architecture. The agency appeared to lack a structured process for testing products within the architecture before placing them into use. The ITRB recommended a centralized test bed which would enable the agency to simulate new functionalities and assess them before placing them into service. One project recently negotiated movement from a cost reimbursable contract to a fixed price contract. The ITRB recommended the agency tie progress payments to accomplishment of specific milestones. The ITRB encountered a project which, after eight years of planning, had yet to define an architecture. The project had come to rely heavily upon the functional program knowledge of the technical contractor, and there were insufficient technical resources involved in crucial technology decision making. The ITRB recommended that the organization establish technical requirements for deliverables, define modular delivery of specified interim products, monitor product delivery, and generally strengthen the role of contract management.
The ITRB Perspective What has been learned from the ITRB's experiences? An agency may be unique, but its systems needs are probably not. As agencies gain experience with mission-critical systems, they can learn valuable lessons from the experiences of others. Set a clear strategic direction Every systems development effort must have a clear strategic direction focused on serving customers and stakeholders. The effort should use capital planning methods to link this direction with the target architecture. Build plans that are intended to be used. Managers uncomfortable with planning may instead delegate planning tasks to staff members or a contractor. The resulting plans may be well written and look wonderful, but no one reads or uses them. Remember that the value of a Strategic Plan rests on the breadth of participation and depth of ownership in it, not in the glossy documentation of the effort. Bring organizational leadership together Organizational leadership should include the CIO's role as technology champion, and should be sure to use performance-based management techniques, capital planning processes, and project management methods. Make realistic promises and deliver on them as soon as possible. When a project demonstrates benefits quickly, it produces an environment of trust. Such an environment encourages collaboration across organizations to support larger efforts later in the development. Manage technology effectively Most agencies require information technology applications that are similar to many others. Help key stakeholders understand how such needs have been met in similar situations and how these similarities offer opportunities to take advantage of reliable, "road-tested" solutions. Use the architecture infrastructure to meet agency business objectives and focus acquisitions of technology in a strategic, comprehensive fashion. *   *   *   *   * "It is not so much that we're afraid of change or so in love with old ways, but it's that place in between that we fear.... It's like being between trapezes.... There's nothing to hold on to." Marilyn Ferguson Futurist Author The ITRB hopes these lessons will be "something to hold on to."	"There will never be a better time to unleash technology, go forth, and do smart things." Arnold Bresnick ITRB Chair